Security panels for use in data centers

ABSTRACT

Disclosed is an integrated data center that provides for efficient cooling as well as efficient wire routing. The data center houses electronic equipment stored in clusters of cabinets. The space inside the data center is shared, such as on a leased basis, between multiple different entities who operate their own electronic equipment. To achieve privacy, security, and cooling air flow, one or more layers of a security paneling surround one or more clusters of cabinets to create a secure interior space. This allows access to only authorized personal, prevents people from viewing into the secure interior space, and allows cooling air flow to pass into the secure interior space to cool the electronic equipment. The security paneling comprises sheets of metal with small apertures. Two or more security panels may be arranged with offset apertures to further prevent viewing of the electronic equipment in the secure interior space.

BACKGROUND 1. Field of the Invention

The disclosed embodiments relate to electronic equipment data centers orco-location facility designs and methods of making and using the same inan environmentally aware manner.

2. Background of the Invention

Data centers and server co-location facilities are well-known. In suchfacilities, rows of electronics equipment, such as servers, typicallyowned by different entities, are stored. In many facilities, cabinetsare used in which different electronics equipment is stored, so thatonly the owners of that equipment, and potentially the facilityoperator, have access therein. In many instances, the owner of thefacilities manages the installation and removal of servers within thefacility, and is responsible for maintaining utility services that areneeded for the servers to operate properly. These utility servicestypically include providing electrical power for operation of theservers, providing telecommunications ports that allow the servers toconnect to transmission grids that are typically owned bytelecommunication carriers, and providing air-conditioning services thatmaintain temperatures in the facility at sufficiently low levels forreliable operation.

There are some well-known common aspects to the designs of thesefacilities. For example, it is known to have the electronic equipmentplaced into rows, and further to have parallel rows of equipmentconfigured back-to back so that each row of equipment generally forcesthe heat from the electronic equipment toward a similar area, known as ahot aisle, as that aisle generally contains warmer air that results fromthe forced heat from the electronics equipment. In the front of theequipment is thus established a cold aisle.

There are different systems for attempting to collect hot air thatresults from the electronics equipment, cooling that hot air, and thenintroducing cool air to the electronics equipment. Theseair-conditioning systems also must co-exist with power andcommunications wiring for the electronics equipment. Systems in whichthe electronics equipment is raised above the floor are well-known, asinstalling the communications wiring from below the electronicsequipment has been perceived to offer certain advantages. Routing wiringwithout raised floors is also known—though not with systematicseparation of power and data as described herein.

In the air conditioning units that are used in conventional facilitysystems, there are both an evaporator unit and a condenser unit. Theevaporator units are typically located inside a facility and thecondenser units are typically disposed outside of the facility. Theseunits, however, are not located in standardized, accessible andrelatively convenient positions relative to the facility should any ofthe units need to be accessed and/or removed for repair or replacement.Further, these units are not themselves created using an intentionallytransportable design.

SUMMARY

The present invention provides an integrated data center that providesfor efficient cooling, as well as efficient wire routing.

In one aspect is provided a facility with an internal area and anexternal area in an external environment for maintaining electronicequipment disposed in a plurality of cabinet clusters in the internalarea at a cool temperature, the facility comprises:

a building that includes an exterior load wall separating the internalarea and the external area;

a plurality of exterior wall openings in the exterior load wall;

a floor within the internal area of the building on which the pluralityof cabinet clusters are disposed;

a plurality of cabinets for holding the electronic equipment therein,the plurality of cabinets positioned in a plurality of rows within eachof a plurality of cabinet clusters so that the electronic equipmentdisposed within the cabinets emit heated air from the cabinets in eachrow of each cabinet cluster toward a central hot air area associatedwith each cabinet cluster;

a plurality of support brackets within each cabinet cluster, disposedalong each of the plurality of rows, that together provide support fordistribution power wiring and conduits, electronic equipment powerwiring and conduits, and communication wiring, wherein a portion of eachof the support brackets is disposed above the plurality of cabinetswithin each cabinet cluster, and wherein some of the distribution powerwiring and conduits string across other cabinets in other cabinetclusters;

a thermal shield supported by the at least some of the plurality ofsupport brackets, the thermal shield providing a contiguous wall aroundthe central hot air area and defining a hot air containment chamber thattraps the heated air within the central hot air area and causessubstantially all the heated air within the central hot air area to riseup within the hot air containment chamber;

a plurality of air conditioning units disposed in the external areaoutside the building that each receive heated air, emit cooled air, andemit vented air, wherein the vented air is released into the externalenvironment;

a warm air escape gap within the building disposed above the hot aircontainment chamber, the warm air escape channel feeding the heated airto the plurality of air conditioning units, the warm air escape gapbeing lowerly bounded by a false ceiling;

cool air ducts within the building that couple the plurality of airconditioning units and the cold aisles, the cool air ducts beingdisposed below the false ceiling and delivering cool air from theplurality of air conditioning units toward the plurality of rows ofcabinets within each of the plurality of cabinet clusters; and

warm air connectors and cool air duct connectors that respectivelyconnect the warm air escape channel and the cold air ducts to theplurality of air conditioning units, and which pass through theplurality of exterior wall openings.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other aspects and features of the present invention willbecome apparent to those of ordinary skill in the art upon review of thefollowing description of specific embodiments of the invention inconjunction with the accompanying figures, wherein:

FIG. 1A illustrates a floor design used in a data center or co-locationfacility according to the present invention.

FIG. 1B illustrates floor-based components disposed over the floordesign according to the present invention.

FIG. 1C illustrates a perspective cut-away view along line c-c from FIG.IA of FIG. IA according to the present invention.

FIGS. 2A-2C illustrate various cut-away perspective views of the thermalcompartmentalization and cable and conduit routing system according tothe present invention.

FIGS. 3A and 3B illustrate modular thermal shields used in the thermalcompartmentalization and cable and conduit routing system according tothe present invention.

FIG. 4 illustrates illustrate a telecommunication bracket used in thethermal compartmentalization and cable and conduit routing systemaccording to the present invention.

FIG. 5A illustrates a top view of a data center or co-location facilityaccording to another embodiment of the present invention.

FIGS. 5B1 and 5B2 illustrate cut-away perspective views of an exteriorand interior portion of the data center or co-location facilityaccording to other embodiments of the present invention.

FIG. 5C shows a first security panel allowing air flow therethrough,according to one exemplary embodiment.

FIG. 5D shows a second security panel allowing air flow therethrough,according to one exemplary embodiment.

FIG. 5E shows a comparison of the first and second panels.

FIG. 5F shows a fenced perimeter around cabinet clusters, according toone exemplary embodiment.

FIGS. 6A and 6B illustrates other telecommunication brackets used in thethermal compartmentalization and cable and conduit routing systemaccording to the present invention.

FIGS. 7A and 7B illustrate a section of a distribution area and the dataarea within a facility according to an embodiment of the presentinvention.

FIG. 8 is a power spine that can also be used with the preferredembodiment.

FIGS. 9A-9E illustrate an air handling unit according to a preferredembodiment.

FIG. 10 illustrates a control system used by the data center.

DETAILED DESCRIPTION OF EMBODIMENTS

The present invention provides data center or co-location facilitydesigns and methods of making and using the same. The data center orco-location facility designs have certain features that will be apparentherein and which allow many advantages in terms of efficient use ofspace, efficient modular structures that allow for efficiency in theset-up of co-location facility and the set-up of the electronicsequipment in the facility, as well as efficient air-conditioning withinthe facility. Each of these features has aspects that are distinct ontheir own, and combinations of these features also exist that are alsounique.

FIG. 1A illustrates a floor design used in a data center or co-locationfacility according to the present invention. The preferred embodimentdiscussed herein uses parallel rows of equipment configured back-to backso that each row of equipment generally forces the heat from theelectronic equipment towards a hot aisle, thus also establishing a coldaisle in the front of the equipment. The cold aisles in FIG. 1A areillustrated at the dotted line block 60, wherein the hot aisles areillustrated at the dotted line block 62. One feature of the presentinvention is the provision for marking the floor 50 to explicitly showthe various areas of the facility. As illustrated, the hot aisle 62 hasa central area 52 that is tiled, painted, taped or otherwise marked toindicate that it is center area of the hot aisle 62, also referred to asa central hot air area. The typical dimensions of the central area 52are typically in the range of 2′-4′ across the width, with a row lengthcorresponding to the number of electronic cabinets in the row. Markingwith tiles is preferable as the marking will last, and tiles that arered in color, corresponding to the generation of heat, have been foundpreferable. Around this center area 52 is a perimeter area 54, overwhich the cabinets are installed. This perimeter area 54 is marked inanother manner, such as using a grey tile that is different in colorfrom the center area 52. Around the perimeter area 54 is an outside area56, which is marked in yet a different manner, such as using a lightgrey tile. The placement of these markings for areas 52, 54 and 56 onthe floor of the facility, preferably prior to moving any equipment ontothe floor, allows for a visual correspondence on the floor of thevarious hot and cold aisles. In particular, when installing cabinetsover the perimeter 54 are, the area that is for the front of the cabinetthat will face the cold aisle, and thus the area for the back of thecabinet for the hot aisle, is readily apparent.

FIG. 1B illustrates floor-based components disposed over the floordesign of the co-location facility according to the present invention.FIG. 1B also shows additional area of the floor, which in thisembodiment is provided to illustrate interaction of the electronicsequipment with the evaporators of the air conditioning units. In theembodiment described with respect to FIG. 1B, certain features areincluded so that conventional equipment, particularly conventional airconditioning equipment, can effectively be used while still creating thedesired air flow patterns of the present invention as described herein.

Before describing the components in FIG. 1B, an aspect of the presentinvention is to isolate the hot air exhaust from the areas that requirecooling as much as possible, and to also create air flows in which theair moves through the exhaust system, into the air conditioning system,through the air conditioning ducts and out to the cool equipment in avery rapid manner. In particular, the amount of circulation establishedaccording to the present invention moves air at a volume such that theentire volume of air in the facility recirculates at least once every I0 minutes, preferably once every 5 minutes, and for maximum cooling onceevery minute. It has been found that this amount of recirculation, incombination with the air flows established by the present invention,considerably reduce the temperature in the facility in anenvironmentally efficient manner.

Cabinets 110 shown in FIG. 1B are placed generally over the sides of theperimeter 54 as described, in rows. Different rows are thus shown withcabinets 110(a-f), with each letter indicating a different row. Alsoincluded within the rows are telecommunications equipment I 70 to whichthe electronics equipment in each of the cabinets 110 connect asdescribed further herein, as well as power equipment I 80, containingcircuit breakers as is known to protect against energy spikes and thelike, that is used to supply power along wires to the electronicsequipment in each of the cabinets I I 0 connect as described furtherherein. Air conditioning units include the evaporator units I 20 (I-6)that are shown being physically separated by some type of barrier fromthe area 56 described previously with respect to FIG. 1A. The condenserunits of the air conditioning system that receive the warmedrefrigerant/water along lines I 22 and are disposed outside the walls ofthe facility are not shown. This physical separation is implemented inorder to establish warm exhaust channel area 240 separate from thephysical space, which warm air area will connect to a separate warm airarea in the ceiling and allow the warm air to flow into the exhaustchannel area 240 and enter into intake ducts of evaporator airconditioning equipment I 20, as will be described. This feature allowsthe usage of conventional evaporator air conditioning equipment that hasair intakes at the bottom of the unit, as well as allows for usage ofdifferent air conditioning equipment types, while still maintaining anefficient airflow throughout the entire facility.

FIG. 1C illustrates a perspective cut-away view along line c-c from FIG.1A of the FIG. 1A co-location facility according to the presentinvention. Additionally, illustrated are the false ceiling I 40 and theactual ceiling I 50, which have a gap that is preferably at least 1.5-3feet and advantageously at least I 5 feet, as the higher the ceiling themore the warm air rises (and thus also stays further away from theequipment in the cabinets I I 0). The false ceiling I 40 is preferablymade of tiles that can be inserted into a suspended ceiling as is known,which tiles preferably have are drywall vinyl tiles, which exhibit agreater mass than many conventional tiles. Also shown are arrows thatillustrate the air flow being centrally lifted upward from the hot airarea containment chamber 210 formed by the thermal shields 400 to thearea between the false ceiling 140 and the actual ceiling 150, and theflow within the ceiling toward the warm exhaust channel area 240, andthen downward into the warm exhaust channel area 240 with the wall 130separating the area 56 and the warm exhaust channel area 240. Also shownare arrows that take cold air from the cold air ducts 310 and insert theair into the cold aisles 60.

Though the arrows in the drawing are directed straight downward, thevents themselves can be adjusted to allow for directional downward flowat various angles. In a preferred embodiment, each of the vents have aremote controlled actuator that allows for the offsite control of thevents, both in terms of direction and volume of air let out of eachvent. This allows precise control such that if a particular area isrunning hot, more cold air can be directed thereto, and this can bedetected (using detectors not shown), and then adjusted for offsite.

FIGS. 2A-2C illustrate various cut-away perspective views of the thermalcompartmentalization and cable and conduit routing system according tothe present invention. In particular, FIG. 2A illustrates a cut awayview of a portion of the hot air area containment chamber 210, whichrests on top of the cabinets 110, and is formed of a plurality of thethermal shields 400 and 450, which are modular in construction and willbe described further hereinafter. Also illustrated are shield brackets500 that are mounted on top of the cabinets 110, and provide for themounting of the shields 400 and 450, as well as an area on top of thecabinets 110 to run power and telecommunications cables, as will bedescribed further herein.

Before describing the cabling, FIG. 2B and FIG. 4 illustrate the shieldand cabling support bracket 500, which is made of structurally soundmaterials, such as steel with a welded construction of the various partsas described, molded plastic, or other materials. Ladder rack supports510, 520, 530, 540 and 550 are attached to back vertical support 502 ofthe shield and cabling support bracket 500 and used to allow ladderracks 610, 620, 630, 640, and 650 respectively, placed thereover asshown. The ladder racks are intended to allow for a segregation of dataand electrical power, and therefore an easier time not only duringassembly, but subsequent repair. The ladder racks are attached to theladder rack supports using support straps shown in FIG. 4 , which aretypically a standard “j” hook or a variant thereof. As also illustratedin FIG. 4 , a support beams structure 506 provides extra support to theladder rack, and the holes 508 are used to secure the shields 400 and450 thereto. Horizontal support plate 504 is used to support the supportbracket 500 on the cabinets 110.

With respect to the cabling and conduit, these are used to provideelectrical power and data to the various servers in the facility.Conduit, containing wiring therein, is used to provide electricity.Cabling is used to provide data. In this system, it is preferable tokeep the electrical power and the data signals separated.

Within the system, ladder rack 610 is used for data cabling on the coldaisle side of the thermal shields 400. Ladder rack 620 is used for anA-source power conduit (for distribution of 110-480 volt power) on thecold aisle side of the thermal shields 400. Ladder rack 630 is used forB-source power conduit (for distribution of 110-480 volt power), whichis preferably entirely independent of A-source power conduit, on thecold aisle side of the thermal shields 400. Ladder rack 640 is used formiscellaneous cabling on the cold aisle side of the thermal shields 400.Ladder rack 650 is used for data cabling on the hot aisle side of thethermal shields 400.

FIGS. 3A and 3B illustrate modular thermal shields 400 and 450,respectively, used in the thermal compartmentalization and cabling andconduit routing system according to the present invention. Both shields400 and 450 are made of a structurally sound material, including but notlimited to steel, a composite, or a plastic, and if a plastic, one thatpreferably has an air space between a front piece of plastic and a backpiece of plastic for an individual shield 400. Shield 400 includes athrough-hole 410 that allows for certain cabling, if needed, to runbetween the hot and cold aisle areas, through the shield 400. Athrough-hole cover (not shown) is preferably used to substantially closethe hole to prevent airflow therethrough. Shield 450 has a 90 degreeangle that allows the fabrication of corners.

It should be appreciated that the construction of the cabinets, theshields 400 and 450, and the shield supports 500 are all uniform andmodular, which allows for the efficient set- up of the facility, as wellas efficient repairs if needed.

Other different embodiments of data center or co-location facilitiesaccording to the present invention also exist. For example, while thefalse ceiling 140 is preferred, many advantageous aspects of the presentinvention can be achieved without it, though its presence substantiallyimproves airflow. Furthermore, the evaporation units for the airconditioning system can also be located outside the facility, in whichcase the chamber 240 is not needed, but hot air from the ceiling can bedelivered to evaporation units that are disposed above the ceiling,which is more efficient in that it allows the warm air to rise. If thecomplete air conditioning equipment is located outside, including theevaporators, the refrigerant/water lines 122 that are used to exchangethe refrigerant/water if the evaporators are disposed inside thefacility is not needed, which provides another degree of safety to theequipment therein.

It is noted that aspects of the present invention described herein canbe implemented when renovating an existing facility, and as such not allof the features of the present invention are necessarily used.

Data Management Center and Integrated Wiring System

In one aspect, the embodiments herein are directed to an overall datamanagement center, including the building itself, interior aspects ofthe building, as well as equipment purposefully located outside yet inclose proximity to the building, which equipment is used for purposes ofproviding both building cooling as well as supplemental power, asdescribed further herein. In one particular aspect, the center core ofthe building that contains the electronics equipment is purposefullycreated in a manner that provides only essential equipment and ductsneeded to provide power, communications, and air flow, while puttinginto periphery areas of the building and outside, other equipment thatcould interfere with the electronics equipment, whether due to thatother equipment requiring extremely high power and/or water or otherliquids to function, all of which can have a detrimental impact on theelectronics equipment.

FIG. 5A illustrates a top view of a portion of a data center orco-location facility 580 according to another embodiment of the presentinvention. In this embodiment, unlike the embodiment shown in FIG. IA- IC, the condenser air conditioning units 800 and heat expulsion chamber900 are all disposed outside of the exterior walls 582 of the facility,as will be described further herein. There is also additional equipmentdisposed outside of the exterior walls 582, including evaporation units59I that feed cooled water along lines 592 to the air conditioning units800 as described further herein, as well as backup diesel generators 594for supplying backup power along a transmission line S96 in the case ofpower outage from remotely supplied power on the national power grid.

FIG. 5B1 illustrates a cut-away perspective view of an exterior andinterior portion (with a 90° rotation for illustrative purposes of theinterior portion) of the data center or co-location facility S80, withthe exterior wall S82 being explicitly illustrated. Shown are two of thecabinet clusters S90-1A and S90-2A, and the corresponding hot air areacontainment chambers 210 and cold air ducts 310, which are respectivelyconnected to the warm exhaust outlets 240-0 and cold duct inlets 310-1.The warm exhaust outlets 240-0 and cold duct inlets 310-I connect toheat expulsion chamber 900 and condenser units 800, respectively.

FIG. 5B2 provides a slightly varied embodiment, in which the cold ductinlets 310-I and warm exhaust outlets 240-0 are each at the same levelas the condenser units 800 and heat expulsion chamber 900, respectively,and the warm exhaust outlets 240-0 contain a 90° angled area, whichallows for better hot air flow into the heat expulsion chambers 900.

Within the facility there are provided distribution areas 584 and 588,as shown in FIG. 5A, as well as data center equipment areas 586, whichequipment areas S86 each contain an array of cabinet clusters 590 (shownin one of the rows as cabinet clusters 590-1, 590-2, 590-3 . . . 590-N),since within each cabinet cluster 590, various cabinets 110 containingdifferent electronic equipment are disposed in rows, thereby allowingeach cabinet cluster 590 to be locked, as well as the cabinets 110within the cabinet cluster 590. It is apparent that three consecutivecabinet clusters, such as 590-1, 590-2 and 590-3 correspond to the threeidentified clusters that are disposed around the associated hot air areacontainment chambers 210 a, 210 b and 210 c in FIG. 1B. As isillustrated, the electronics equipment within each cabinet 110 of acabinet cluster 590 is connected in a manner similar to that asdescribed in FIGS. 2A-2C previously.

It is noted that the cabinet cluster may have an actual physicalperimeter, such as a cage built with fencing that can be locked andstill permits airflow therethrough, or alternatively need not have anactual physical perimeter, in which case the orientation of the cabinets110 and corresponding other structures as described previously withreference to FIGS. 1A-1C can also define this same space.

In some embodiments, the cabinet cluster may comprise sensitiveelectronic equipment that is to be kept out of view from unauthorizedpersons. In this instance, the physical perimeter may comprise fencingor paneling that permits airflow while also blocking view of the cabinetcluster.

FIG. 5C shows a first security panel allowing air flow therethrough,according to one exemplary embodiment. The first security panel 5000 iscomprised of a panel 5002 formed from a sheet metal or other suitablematerial. The panel 5000 includes a plurality of hole arrays 5010. Thehole arrays 5010 are made up of holes 5012 that allow air flow from oneside of the panel 5000 to the other.

FIG. 5D shows a second security panel allowing air flow therethrough,according to one exemplary embodiment. The second security panel 5500 iscomprised of a panel 5502 formed from a sheet metal or other suitablematerial. The panel 5500 includes a plurality of hole arrays 5510. Thehole arrays 5510 are made up of holes 5512 that allow air flow from oneside of the panel 5500 to the other.

The hole arrays 5010, 5510 are positioned on each of the panels 5000,5500 so that when the panels are aligned and parallel with one another,the hole arrays 5010, 5510 do not overlap. In this manner, the air flowmay still travel through the panels 5000, 5500, but a direct line ofsight from the outside to the inside of the physical perimeter isprevented.

Accordingly, returning to FIG. 5C, the first panel 5000 is formed with afirst vertical dimension 5100 and a first horizontal dimension 5200.From a top of the panel 5000, a first row of hole arrays 5010 begins ata first-row dimension 5110 from the top of the panel 5000. A second rowof hole arrays 5010 begins at a second-row dimension 5120 from the topof the panel 5000. A third row of hole arrays 5010 begins at a third-rowdimension 5130 from the top of the panel 5000. A fourth row of holearrays 5010 begins at a fourth-row dimension 5140 from the top of thepanel 5000. A fifth row of hole arrays 5010 begins at a fifth-rowdimension 5150 from the top of the panel 5000.

The columns of the hole arrays 5010 may be positioned such that a firstcolumn of hole arrays 5010 begins at a first-column dimension 5210 fromthe left side of the panel 5000, a second column of hole arrays 5010begins at a second-column dimension 5220 from the left side of the panel5000, and a third-column of hole arrays 5010 begins at a third-columndimension 5230 from the left side of the panel 5000.

As shown, the dimensions of the positions of the rows of hole arrays5010 in the first panel 5000 leave spaces adequate to line up with holearrays 5510 of the second panel. In one exemplary embodiment, the firstvertical dimension 5100 of the panel 5000 is 43 inches. The first-rowdimension 5110 may be 2.296 inches, the second-row dimension 5120 may be11.296 inches, the third-row dimension 5130 may be 20.296 inches, thefourth-row dimension 5140 may be 29.296 inches, and the fifth-rowdimension 5150 may be 38.296 inches. The first horizontal dimension 5200may be 47.687 inches, the first-column dimension 5210 may be 3.219inches, the second-column dimension 5220 may be 18.219 inches, and thethird column dimension 5230 may be 33.219 inches. Of course, thesedimensions are exemplary, and may be changed based on the size, number,and shape of the holes 5012 in the hole array 5010, the shape of thehole array 5010, the amount of air flow required, etc.

In FIG. SD, the second panel 5500 is formed with a second verticaldimension 5600 and a second horizontal dimension 5700. From a top of thepanel 5500, a first row of hole arrays 5510 begins at a first-rowdimension 5610 from the top of the panel 5500. A second row of holearrays 5510 begins at a second-row dimension 5620 from the top of thepanel 5500. A third row of hole arrays 5510 begins at a third-rowdimension 5630 from the top of the panel 5500. A fourth row of holearrays 5510 begins at a fourth-row dimension 5640 from the top of thepanel 5500.

The columns of the hole arrays 5510 may be positioned such that a firstcolumn of hole arrays 5510 begins at a first-column dimension 5710 fromthe left side of the panel 5500, a second column of hole arrays 5510begins at a second-column dimension 5720 from the left side of the panel5500, and a third-column of hole arrays 5510 begins at a third-columndimension 5730 from the left side of the panel 5500.

As shown, the dimensions of the positions of the rows of hole arrays5510 in the second panel 5500 leave spaces adequate to line up with holearrays 5010 of the first panel. In one exemplary embodiment, the secondvertical dimension 5600 of the panel 5500 is 43 inches. The first-rowdimension 5610 may be 5.796 inches, the second-row dimension 5620 may be15.796 inches, the third-row dimension 5630 may be 24.796 inches, andthe fourth-row dimension 5640 may be 33.796 inches. The first horizontaldimension 5700 may be 47.687 inches, the first-column dimension 5710 maybe 3.219 inches, the second-column dimension 5720 may be 18.219 inches,and the third column dimension 5730 may be 33.219 inches. Of course,these dimensions are exemplary, and may be changed based on the size,number, and shape of the holes 5012 in the hole array 5010, the shape ofthe hole array 5010, the amount of air flow required, etc.

FIG. 5E shows a comparison of the first and second panels. Here, therows of the hole arrays 5010 of the first panel 5000 can be seen to beoffset from the rows of hole arrays 5510 of the second panel based onthe exemplary dimensions provided above. Of course, other configurationsare possible. For example, instead of each array extending horizontallyand the rows being offset vertically, the arrays may extend verticallywith columns being offset horizontally. The arrays may form other shapesinstead of the rectangular pattern shown herein, such as square,circular, triangle, or any other number of shapes. The holes in thearrays are shown to be slightly offset so as to be more densely disposedin the arrays, but other hole patterns within the array as well as holeshapes may be included. For example, circular, square, or triangularholes may be aligned vertically and horizontally in the arrays. In otherembodiments, the holes may be hexagonal and form a honeycomb shape inthe array.

FIG. 5F shows a fenced perimeter around cabinet clusters, according toone exemplary embodiment. As shown in FIG. 5F, cabinet clusters 590 aresurrounded walls formed by the first panel 5000 and the second panel5500 to form a secure perimeter. The prevents the cabinet clusters 590from being seen from the outside. In this embodiment, the wall formedfrom the first panels 5000 form an inner perimeter, and the wall formedfrom the second panels 5500 for an outer perimeter. However, the panels5000, 5500 may be switched. Further, additional panels may be added ontop of the panels shown in FIG. 5F to extend from floor to ceiling.Because the hole arrays of the panels are offset from one another, it isdifficult to see through to the cabinet clusters, even though air flowis permitted through the panels.

The manner in which the distribution power wires and conduits,electronic equipment control wires and conduit, data cabling, andmiscellaneous cabling is distributed to the cabinet clusters 590 fromone of the distribution areas 584 or 588 will be described furtherhereinafter. As shown in FIG. 5A, telecommunications, and powerdistribution equipment, further described herein, is used to then feedthe appropriate signals and power to the telecommunications equipmentand power equipment that is stored within each cabinet cluster 590(i.e., telecommunications equipment 170 and power equipment 180described in FIG. 1B). The manner in which the distribution power wiresand conduits, electronic equipment control wires and conduit, datacabling, and miscellaneous cabling is distributed to the cabinetclusters 590 from one of the distribution areas 584 and 588 will bedescribed further hereinafter.

The array of cabinet clusters 590, and the density of the cabinets 110and the electronics equipment therein, require substantial amounts ofpower and transmission capacity, which in turns requires substantialamounts of wiring, particularly for power. As described herein, as aresult there is described an improved telecommunication bracket 600,which substantially rests over each of the cabinets in the cabinetclusters 590, in order to more easily accommodate the distribution powerwires and conduits, as well as telecommunication wires and conduits, aswell as control wires and conduits, that are then distributed from thedistribution areas 584 and 588 to the telecommunications equipment 170and power equipment 180 that is within each of the different cabinetclusters 590. As shown in FIG. 5A and FIG. 8 , the distribution area 588contains PDU's 598, described in further detail elsewhere herein, andthe distribution area 584 contains transformers to step down the powergrid power that is normally at 12477 volts to a 480 volt level, fortransmission of 480 volt power to the PDU's 598. Also withindistribution area 584 are uninterruptable power supplies in case anoutage of power from the power grid occurs, as well as equipment fortesting of the various power equipment that is conventionally known.

While FIG. 1B illustrates one configuration of equipment with thecabinet cluster (with the telecommunications equipment 170 and the powerequipment 180 within the center of a row), FIG. 7A also shows analternative configuration of equipment for a cabinet cluster 590, whichstill contains the same cabinets 110, telecommunication equipment 170and power equipment 180. In particular, rather than having the powerequipment 180 centrally located within a row, in this alternateconfiguration the power equipment 180 is disposed at an end of each ofthe rows that are within a cabinet cluster 590. The telecommunicationequipment, within this embodiment, can be located anywhere within therow of cabinets 110, within whichever one of the cabinets 110 makes mostsense given the usage considerations for that cabinet cluster 590.

In another variation of the FIG. 7A embodiment, the power equipment 180,instead of being somewhat separated from the cabinets 110 within acluster 590, instead abut right next to one of the cabinets 110. This,along with the doors 593 shown in FIG. 7A then being attached betweenadjacent power equipment at the end of the cabinet row instead of at theend cabinet, keep all the equipment in a tightly configured space. Inany of the embodiments shown, whether FIG. 1C, 7A or as described above,the thermal shield 400 that creates the hot air area containment chamber210 above the cabinets, coupled with the doors that seal off the areabetween the rows of cabinets 110 within a cluster 590, provide anenvironment that prevents the hot air within the hot air area 52 fromescaping out into the main data center floor, and ensures that the hotair instead travels up through the hot air area containment chamber 210and into the gap disposed between the false ceiling 140 and the actualceiling 150.

Within equipment area 586 is thus established an array of cabinetclusters 590, which cabinet clusters align with each other to allow forthe overhead stringing of telecommunications and power wiring asdescribed herein. Within each cabinet cluster 590, as also shown in FIG.1B, is telecommunications equipment 170 to which the electronicsequipment in each of the cabinets 110 connect, as well as powerequipment 180 used to connect the electronics equipment to power. Thearray of cabinet clusters 590, each also containing brackets, such asbrackets 500 or 600, as described herein. For a larger size data centeras illustrated in FIG. SA that contains a very large array of cabinetclusters 590, brackets 600 are preferable, as they allow for additionalconduit support areas. These brackets 600, discussed further herein withrespect to FIGS. 6A and 6B, contain ladder racks 510, 520, 530 and 540that are used for stringing power and telecommunication wiring withineach cabinet cluster 590, as well as contain additional vertical supportwith conduit clamps that are used to hold power and telecommunicationlines that pass from each cabinet cluster 590 to other centraltelecommunication and power distribution areas, as discussed furtherherein, as well as to hold power and telecommunication lines that passover certain of the cabinet clusters 590 in order to be strung to otherof the cages areas 590. Still further, these same brackets 600, beingpreferably mounted over the cabinets 110, and at least having asignificant portion of the bracket disposed over the cabinets 110, areused to mount the thermal shield within the cabinet cluster 590, thethermal shield providing a contiguous wall around the central hot airarea of the cabinet cluster 590, and defining a warm exhaust channelthat traps the heated air within the central hot air area and causessubstantially all the heated air within the central hot air area to riseup within the warm exhaust channel. These brackets 600 also preferablyspan from the top of the cabinets 110 to the bottom of the false ceiling140 to provide further stability.

It is apparent that the power and telecommunication lines that pass fromeach cabinet cluster 590 to other more central telecommunication andpower distribution areas will necessarily pass, in some instances, overother cabinet clusters 590. Since the vertical support 610 with conduitclamps 620 are above the ladder racks 510, 520, 530 and 540 for each ofthe brackets 600, as well as above each of the cabinets 110, this allowsfor long runs of power and telecommunication lines that pass from eachcabinet cluster 590 to other more central telecommunication and powerdistribution areas to exist without interfering with the wiring thatexists within each cabinet cluster 590. Furthermore, by creating asufficient area of vertical support and conduit clamps, it is thenpossible to run additional power and telecommunication lines fromcertain cabinet clusters 590 to other more central telecommunication andpower distribution areas without having to re-work existing wiring. Thismakes expansion much simpler than in conventional designs.

FIGS. 6A and 6B illustrate in detail two different embodiments of thetelecommunication bracket 600 referred to above that is used in thethermal compartmentalization and cable and conduit routing systemaccording to the present invention. This bracket 600 serves the samepurpose as the bracket 500 illustrated and described previously withrespect to FIG. 4 , and as such similar parts of the bracket 600 arelabeled the same and need not be further described herein. This bracket600, however, additionally provides additional vertical support 610 thatallows for the running of additional wiring and conduits.

In FIG. 6A, this additional vertical support 610 includes conduit clamps620 that allow the clamping of the additional conduits to the additionalvertical support 610.

In FIG. 6B, the bracket 600A has in addition to the vertical support 610a support beam 506A (which extends upwards from the support beam 506shown in FIG. 4 ), and racks 630, 632, 634, 636, 638, and 640therebetween. Each of the racks 630, 632, 634, 636, 638, and 640 haveroom for at least 4 different 4″ conduits to run wiring or cablingtherethrough. Whether the conduit clamps or additional conduit racks areused, both provide for conduit holding, and holding of the wires orcables within the conduits.

In both the brackets 600 of FIGS. 6A and 6B, the additionalwiring/conduit is distribution power wires and conduits and otherwire/conduit for control uses, for example. As explained hereafter, thedistribution power wires and conduits can run from various powerequipment units 180 disposed in each of the cabinet clusters 590 tovarious other high power distribution units (PDUs) 598 disposed withinthe distribution area 588, as shown in FIG. 7A.

FIG. 7A also illustrates the distribution of power PDUs 598 within asection of the distribution area 588 to power equipment 180 in an endcabinet cluster 590-1 within a section of the data equipment center area586 via distribution power wires and conduit (one shown as 597). Inparticular, as is shown, distribution power wires and conduit goes fromeach of the PDUs 598A and 598B to the power equipment unit 180A withinthe end cabinet cluster 590-1, and distribution power wires and conduitalso goes from both the PDUs 598A and 598B to the power equipment unit180B within the end cabinet cluster 590-1, so that redundant power canbe provided to the electronic equipment within each row. Since power isprovided to each piece of power equipment 180 from two differentsources, these power equipment units can also be called redundant powerpanels, or RPP's. In addition, distribution power wires and conduit gofrom each of PDUs 598A and 598B over the end cabinet cluster 590-1 tofurther cabinet clusters 590-2, 590-3 to 590-N. The array of cabinetclusters 590 are aligned as shown in FIG. 5A so that the brackets 600 indifferent cabinet clusters 590 nonetheless can together be used tostring distribution power wires and conduit and other wires/fibers withconduits as needed.

In a preferred configuration of the power equipment 180 shown in FIG. 7Aprovides redundant 120 volt AC power from each RPP 180 to the electricalequipment in each of the cabinets 110 within the row of the cabinetcluster 590. Within the RPP 180 are circuit breakers as is known toprotect against energy spikes and the like, as well as energy sensorsassociated with each circuit so that a central control system, describedhereinafter, can monitor the energy usage at a per circuit level. In atypical implementation, there are 42 slot breaker panels that areassociated each with 120c/208v power that is then supplied to each ofthe electronic components as needed, in wiring that uses one of theladder racks 630 or 640 as discussed previously to the necessary cabinet110. Of course, other power configuration schemes are possible as well.

In a preferred configuration for a module of cabinet clusters 590, asschematically shown in FIG. 7B, there are three different PDUs 598 thateach receive 480vAC 3-phase power and provide 120vAC 3-phase powerservice to each of 8 different RPPs 180 via the distribution power wiresand conduits. This allows, for a completely used module, 6 differentcabinet clusters 590 to be serviced from 12 RPP's 180, two in each cage,and 3 different PDU's 598. By providing redundancy of both RPP's 180(×2) and PDUs 598 (×3), this allows for maximum power usage of thevarious components with sufficient redundancy in case any one of thePDU's 598 or any circuit on an RPP 180 fails.

A lock-related aspect of the present invention with respect to the RPPs180 as well as the PDU's 598 is that since there are three circuits fromthe PDU's t the RPP's, within a dual RPP each side of the cabinet willhave separate lock, such that all locks of a particular circuit can beopened by the same key, but that key cannot open locks of any of theother two circuits. This is an advantageous protection mechanism, as itprohibits a technician from mistakenly opening and operating upon adifferent circuit than a circuit he is supposed to service at that time.

FIG. 8 shows a power spine 599 that can also be used with the preferredembodiment to provide power from the power grid to each of the PDU's598. As illustrated, rather than running the power spine through theroof as is conventionally done, in this embodiment the power spine 599is run along a corridor within the distribution area 588 that channelsall of the main building wiring and electrical components. Thisadvantageously reduces stress on the roof and building structure, as theweight of the power spine and related components are supportedinternally within the corridor structure as shown.

Data Center Air Handling Unit

Another aspect of the data center is the air handling unit that providesfor efficient cooling.

As is illustrated in FIGS. 5A and 5B1-2, one condenser unit 800 ispaired with one heat expulsion chamber 900, and each are preferablyindependently movable. As is further illustrated, the condenser units800 are built to a size standard that allows for transport along USstate and interstate highways. Further, the heat expulsion chamber 900is preferably sized smaller than the condenser unit 800, but stillhaving dimensions that allow for transport using a semi- trailer. Whentransported to the facility 500, the condenser unit 800 is first placedinto position, as shown here on posts 588, but other platforms can alsobe used. As shown in this embodiment, the heat expulsion chamber unit900 is placed over the condenser unit 800, though other placements, suchas adjacent or below, are also possible. Connections of power conduit,miscellaneous cabling, and water needed for proper operation of thecondenser units 800 and expulsion chamber 900 is preferably made usingeasily attachable and detachable components.

With this configuration, the units 800 and 900 are located instandardized, accessible and relatively convenient positions relative tothe facility 580 should any of the units 800/900 need to be accessedand/or removed for repair or replacement. Further, these units 800/900are themselves created using an intentionally transportable design.

FIGS. 9A-9E provide further details regarding the condenser unit 800 andits paired heat expulsion chamber 900. In particular, as shown, the airconditioning apparatus includes the condenser unit 800 and its pairedheat expulsion chamber 900. The heat expulsion chamber 900 receivesheated air, and emits vented air, and the vented air is released intothe external environment, while the condenser unit 800 emits cooled air.

The heat exchange unit 900 contains an exhaust fan 910, controlled by aVFD fan control and I/O signals block 1330 shown in FIG. 10 , that emitsheat from the heated air as the vented air, thereby allowing return airto pass through a return damper 920, which return damper 920 has areturn damper actuator associated therewith.

The condenser unit 800 includes an outside air inlet 810, and hasassociated an outside air damper 812, thereby allowing outside air topass therein. This outside air damper 812 is preferably coated with aneoprene seal to prevent pollution particles from passing through thedamper 812 when in a closed position, as well as contains aspring-loaded mechanism closing lever that will automatically close theoutside air damper 812 upon a removal of power, so that outside air isprevented from intake before backup generators 594 have to start, sinceafter a power-grid power failure condition, before the back-upgenerators start, uninterruptable power supplies will supply buildingpower, giving a period for the outside air damper 812 to close.

A filter chamber 820, which includes an air intake area 822 coupled tothe heat expulsion unit 900 and the outside air inlet 810, isconfigurable, via the AHU control system 1000, described hereinafter, toreceive the return air, the outside air, as well as a mixture of thereturn air and the outside air, the filter chamber resulting in filteredair. In a preferred implementation of the filters 824 within the filterchamber 820 are included a MERV 7 screen filter 824A with a MERV 16 bagfilter 824B therebehind, which allows replacement of the screen filter824A without replacement of the bag filter 824B, and vice-versa.

The condenser unit 800 includes an air cooling area 830 over which thefiltered air passes to create the cooled air. For ease of nomenclature,all of the air within the air cooling area 830 is referred to asfiltered air, and only upon emission from the condenser unit is itreferred to as cooled air. That notwithstanding, it is understood thatalong various stages of the air cooling area 830, the filtered air willget progressively cooler in temperature.

The air cooling area 830 of the condenser unit 800 includes a directcooling coil 840 filled with a gas for direct expansion, such as R134gas, over which the filtered air passes, the gas being circulatedthrough a condenser 842 disposed in another area of the condenser unithousing, but still in the external area, outside of the building.

The air cooling area 830 also includes an indirect cooling coil 850filled with cooled water over which the filtered air passes, the cooledwater being circulated through an evaporation unit 590 also disposed inthe external area, via a water line 592 as shown in FIG. 5A Optionally,though not shown, another coil that is cooled by a chiller could beincluded.

Also shown in FIGS. 9A-9E is that the air cooling area also has anevaporator 860 that provides a water wall through which the filtered aircan pass. An evaporator bypass 862 allows all or some of the filteredair to bypass the evaporator 860, and a bypass damper 880 is opened toallow 100% bypass of the evaporator 860, in which case the evaporatordamper 890 is then fully closed. Filtered air can also be partiallybypassed, or all go through the evaporator 860, depending on thepercentage opening of each of the dampers 880 and 890.

Also within the air cooling area 830 is a fan 870, shown as a fan arrayof multiple fans, operable to push the filtered air through the aircooling area 830, as well as an outlet damper 880 controllable by anactuator and operable to control an amount of the cooled air deliveredfrom the air cooling area 830.

As shown and mentioned previously the heat exchange unit 900 iscontained within a first housing, and the condenser unit 800 iscontained within a second housing.

Furthermore, and with reference to FIG. 10 , overall air conditioningsystem for the data center 500 includes a control system 1000. Thecontrol system 1000 contains an air handling unit (AHU) and powercontrol system computer 1100, which is operable to automatically controleach of the exhaust fan 910, the return damper actuator, the outside airdamper actuator, the condenser 842, the bypass damper actuator, the fan870, and the outlet damper actuator.

Air Handling Control System

As referenced previously, and shown explicitly in FIG. 10 , the datacenter 580 includes a control system 1000. The control system includesan air handling unit (AHU) and power control system (PCS) computer 1100,which as shown obtains signals from many different units, and sendssignals to many different units, based upon various software routinesrun by the AHU/PCS computer 1100. These routines can be integrated witheach other, as well as be discrete modules which operate on their own,or a combination of both.

A significant aspect of the present invention is the placement ofsensors that can monitor for each/all of temperature, pressuredifferential, airflow, and humidity. Sensors that monitor thesedifferent aspects are placed in different locations throughout the datacenter.

In particular, having temperature sensors inside the thermal shield 400(preferably redundant ones at the two ends and the middle of the clusterat least), and at different levels (such as at the middle and top of acabinet 110, as well as at the middle and top of the thermal shield400), as well as in stratified locations in the gap between the falseceiling 140 and the actual ceiling 150 (spaced at intervals of between2-4 feet, as well as outside the thermal shield area, at the outside ofcabinets in the cold aisles, allows for precise temperature gradientinformation throughout the facility.

Humidity sensors are helpful to have at locations that are the same asthe temperature sensors, though fewer are needed, as humidity data neednot be as precise for overall control of the building thermalenvironment.

Pressure differential sensors are also preferably located, redundantly,in a number of different areas. These include within the thermal shieldbelow the false ceiling 140, outside the thermal shield below the falseceiling 140, at different locations in the gap between the false ceiling140 and the actual ceiling 150 (spaced at intervals of between 2-4feet), at various locations within the cold aisle ducts 310,particularly a header plenum that has a main cold air area to which manyof the different condenser units connect, shown best along 310-I in FIG.5B2 and then distribute cool air to the cooling ducts 310 that form thecold aisles. This allows for sensing of the pressure at variouslocations, and in particular within the hot air containment chamber 210,outside the hot air containment chamber 210 above the cabinets 110,within the gap between the false ceiling and the actual ceiling 150, andwithin the cold aisle ducts. This allows for modification of the airhanding units 800/900 by the control system 1100. Overall pressurecontrol between the hot air containment chamber 210, the cold aisle, andthe gap between the false ceiling and the actual ceiling 150 is achievedby adjusting the air handling units 800/900 so that the pressure ismaintained in these different areas within a predetermined range of eachother, for example. This also allows for running the facility at apositive pressure differential when outside air is used, at ranges of 1%to 6%, such that as in essence the building breathes out.

Airflow sensors are also preferably located in each of the areas wherethe pressure differential sensors are noted as being required, in orderto ensure that the airflow is stable, as amounts of airflow that are toogreat, just as pressure differentials that are too great, can adverselyaffect the electronic equipment.

Areas where these differentials occur the most in the embodimentsdescribed herein are at the barrier caused by the thermal shield 400within each cabinet cluster 590, between the false ceiling and the gapthereover, since heated air from each of the different hot aisle areas210, associated with each cabinet cluster 590, vent to this large gaparea.

Signals from these sensors, as shown by Temperature, PressureDifferential, Airflow, and Humidity Sensor Control and input/output(1/0) signals (block 1310) can then be used to provide damper actuatorcontrol (block 1320), VFD fan control and 1/0 signals (block 1330),evaporator control and 1/0 signals (block 1340), condenser control and1/0 signals (block 1350), evaporator control and 1/0 signals (block1360), and optionally chiller control and 1/0 signals (block 1370).Within the Damper actuator control block is included the dampersassociated with the cold aisle ducts, which dampers can be automaticallyadjusted to fully open, fully closed, or in-between amounts based uponsensing of the current conditions, as described previously.

Still furthermore, the AHU/PCS computer 1100 also monitors powerconsumption and power production, depending on the devices, to assessoverall power usage. As such, electrical energy monitor sensors withinthe RPP 180 are operated upon by the RPP control and 1/0 signals block1410, and provide an indication of the power usage of the electronicsdevices in the cabinets

The PDU 598 is monitored, as is known, and operated upon by the PDUcontrol and 1/0 signals block 1420. Power load control and 1/0 signalsblock 1430 provides monitoring of the transformers and uninterruptablepower supplies within the distribution area 584. Backup generatorcontrol and 1/0 signals block 1440 is used for the control of the backupgenerator 594, whereas telecommunication control and 1/0 signals block1450 is used for the control of the telecommunications equipment.Equipment load control and 1/0 signals block 1460 controls and monitorsenergy consumption of other equipment within the data center facility.

The above control blocks can contain software written to both act uponinput signals obtained from other sensors or other units, and ensurethat the various different units operate together. The usage of the term1/0 signals is intended to convey that for any of the associatedsensors, actuators for dampers, VFD for fans, and other mechanisms, thatdepending on the model used, such devices may output signals, inputsignals or both.

It is also noted that what occurs with one device will alter which otherdevices operate. Thus, for example malfunction of a particular circuitin an RPP 180 will cause the AHU/PCS computer 1100 to switch over to theredundant circuit in the same RPP 180 until that circuit is fixed.

It is particularly noted that the above system can monitor and controlfor certain situations that are particularly significant for datacenters. For example, the air flow patterns that are caused, with theinclusion of the false ceiling 140 as shown in FIG. 1C, requireassessment of high and low pressure areas. The AHU/PCS computer 1100 canmonitor for this, and as a result maintain a balance, thus ensuring thatfans and other components that are within the electronics equipmentstored in the cabinets 110 isn't damaged.

Also shown in FIG. 10 are building cabinet cluster and cage lock sensorsblock 1510. This allows for the detection of which cabinet clusters 590,as well as which cabinets 110, are open, based upon sensors that areplaced at each of these areas.

Fire and roof water detection leak sensors module 1520 is also shown, asthis can be used in conjunction with known systems, and interfaced withthe other blocks referred to herein, to ensure that if a fire or leak isdetected, that appropriate shut down of equipment in the preferredsequence to avoid damage is done.

Although the present invention has been particularly described withreference to embodiments thereof, it should be readily apparent to thoseof ordinary skill in the art that various changes, modifications andsubstitutes are intended within the form and details thereof, withoutdeparting from the spirit and scope of the invention. Accordingly, itwill be appreciated that in numerous instances some features of theinvention will be employed without a corresponding use of otherfeatures. Further, those skilled in the art will understand thatvariations can be made in the number and arrangement of componentsillustrated in the above figures.

What is claimed is:
 1. A facility for maintaining electronic equipmentat a cool temperature, the facility comprising: a plurality of cabinetclusters, each cabinet cluster comprising a plurality of cage cabinetspositioned in a back-to-back configuration in two separated rowsdefining a hot aisle therebetween, the cage cabinets housing electronicequipment that emits heat towards the hot aisle, and the plurality ofcabinet clusters being disposed in a cabinet array defining at leastcommon cold aisle between cabinet clusters of the plurality of cabinetclusters, the at least one common cold aisle being parallel and alignedwith the hot aisle, air flow within the facility being directed to passinto the at least one cold aisle, over the electronic equipment, andinto the hot aisle; a security perimeter surrounding the plurality ofcabinet clusters, the perimeter comprising: an interior wall formed fromfirst panels, the first panels being configured to allow air to flowtherethrough; a second wall formed from second panels, the second panelsbeing configured to allow air to flow therethrough, the second panelsbeing aligned with the first panels to prevent a view of the cabinetcluster from outside the perimeter.
 2. The facility of claim 1, whereinthe first panels comprise first apertures to allow air to flowtherethrough.
 3. The facility of claim 2 wherein the first apertures areformed in at least one first hole array on the first panels.
 4. Thefacility of claim 3, wherein the at least one first hole array has arectangular shape.
 5. The facility of claim 4, wherein the rectangularshape extends horizontally.
 6. The facility of claim 3, wherein thesecond panels comprise second apertures to allow air to flowtherethrough, and the second apertures are formed in at least one secondhole array on the first panels.
 7. The perimeter of claim 6, wherein thesecond panels are aligned with the first panels so that the at least onesecond hole array does not overlap with the at least one first holearray, preventing a view of the cabinet cluster from outside theperimeter.
 8. A perimeter for a cabinet cluster storing electronicdevices, the perimeter comprising: an interior wall formed from firstpanels, the first panels having one or more first panel aperturesconfigured to allow air to flow therethrough; a second wall formed fromsecond panels, the second panels having one or more second panelapertures to allow air to flow therethrough, the second panel aperturesnot aligned with the first panel apertures to prevent a view of thecabinet cluster from outside the perimeter.
 9. The perimeter of claim 8,wherein the first panels and the second panels are formed from sheetmetal.
 10. The perimeter of claim 8 wherein the first panel aperturesare formed in at least one first hole array on the first panels and thesecond panel apertures are formed in at least one second hole array onthe second panels.
 11. The perimeter of claim 10, wherein the at leastone first hole array has a rectangular shape.
 12. The perimeter of claim11, wherein the rectangular shape extends horizontally.
 13. Theperimeter of claim 8, wherein the first panels and second panels extendfrom a floor to a ceiling.
 14. The perimeter of claim 10, wherein thesecond panels are aligned with the first panels so that the at least onesecond hole array does not horizontally overlap with the at least onefirst hole array, preventing a view of the cabinet cluster from outsidethe perimeter.
 15. The perimeter of claim 8, wherein the perimeter isrectangular shaped and generally parallel to the cabinet cluster. 16.The perimeter of claim 8, wherein the view is a direct line of sightfrom outside the perimeter to inside of the perimeter.
 17. The perimeterof claim 14, wherein the view is a direct line of sight from outside theperimeter to inside of the perimeter.
 18. The perimeter of claim 10,wherein the at least one first hole array is hexagonal to form ahoneycomb shape in the array.
 19. The perimeter of claim 8, wherein theperimeter entirely surrounds the cabinet cluster.
 20. The perimeter ofclaim 8, wherein the interior wall and exterior wall comprises lockablefencing to permit access to an interior of the perimeter.